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.CAPTURE, MANIPULATION AND INCORPORATION 

Priority is claimed from provisional application no. 60/204,546 filed 5/16/00 
incorporated by reference herein. wH,j«to niea i/lb/UU, 

BACKGRO UND OF THF LNVF.NTTOiV 

1. Field of the Invention 

The invention relates generally to game machines for playing realistic 
video and/or computer games. In particular, the invention is directed to a game 
machine having the abilities to capture and/or transfer an image, to manipulate 
and edit the captured image, to incorporate the captured and/or manipulated 
captured image into a game that is being played on the game machine, and to 
map the captured image onto an animated player in the game so that the player 
has characteristics, e.g., facial features, that are derived from the captured 



10 image 



2. Descriptio n of Relatpd Art 

Video and computer game machines have been known for quite some 
time. Typically, these game machines include a game machine housing, a 
processing unit, associated hardware for running a gaming program, and a 
15 display for displaying images of the game. The gaming program is typically 
contained in a game program memory, such as, for example, a semiconductor 
memory (e.g., ROM, EPROM, etc.) that is part of a removable cartridge or 
other removable memory device. By storing the gaming program in a 
removable cartridge, the user is allowed to conveniently and easily change the 
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game being played by simply exchanging the cartridge with one containing a 
different game. Examples of such game machines are the "N64" and "Game 
Boy" products manufactured and sold by Nintendo Corporation. 

Conventional game machines of the type described above execute the 
5 game that is contained in the game program memory of a particular removable 
cartridge or other removable memory device. The images displayed during 
game play are limited to those that are contained in a memory of the removable 
game cartridge. While the images may change based on various inputs received 
from the user, these images of these conventional game machines are 

10 predefined and are not subject to any manipulation, or editing or variation by 
the user, other than those predetermined changes that occur based on the 
various situations encountered during game play. 

The limited predetermined images available for use in a game machine 
restricts the ability of the user to enjoy the machine. It may be more enjoyable 

15 . for the user to have a more interactive relationship with the game program and 
the game machine. To that end, what is needed is a game machine that enables 
the user to define, manipulate, edit and incorporate images into the game being 
played. This ability would allow the user to personalize the game, including the 
animated game players that appear in the images of the game. 

20 Additionally, it would be beneficial to provide the user with the ability to 

capture images for use with a game, and store captured images for use with 
various other features of the game machine. Image capture and manipulation 
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technologies, e.g., digital cameras, are well known and exist in many forms. 
Typically, these image capturing and manipulation devices are very costly and 
are cumbersome to use, especially in the context of a personal game machine. 
For example, there are many different types of video cameras and digital still 
cameras available for capturing images. However, video cameras and digital 
still cameras are typically very expensive (especially as compared to portable 
game machines), and would not be used as a toy or game for children. It is also 
very complicated to manipulate the images captured by these cameras, 
especially for use in a game environment. 

There are also various computer based applications that provide a user 
with the ability to receive a digital image and to manipulate the digital image. 
There are also numerous computer programs that enable a user to draw an 
image or to create animation. However, as set forth above, all of these known 
image capturing and manipulation technologies are ill suited for portable game 
use, and are typically prohibitively expensive in the context of gaming 
applications. 

Prior video game systems have been developed that capture a real-life 
image of an individual's face and insert that facial image onto an animated 
game player. Such a system is disclosed in U.S. Patent No. 4,7 10,873 ('873 
Patent), that is entitled "Video Game Incorporating Digitized Images of Being 
Into Game Graphics." However, the video game system with image capture 
that is disclosed in the '873 Patent does not: map an external facial image onto 
a head of animated game player; apply a two-dimensional facial image onto a 
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three-dimensional (3-D) graphical image of a head; generate a 3-D graphical 
image having a personalized facial image that turns and otherwise moves in 
three dimensions during game play; and does not provide an image editor to 
manipulate and modify an external image inserted into a video game. 
5 Accordingly, the prior image capture systems for game machines lack functions 
and features, e.g., 2D and 3D editing, that would facilitate the enjoyment of a 
game play. 

SUMMARY OF THE INVENTION 

To overcome the above-noted and other deficiencies of game systems, 
10 and to improve the enjoyment of game system users, the instant invention 
provides a game machine that maps external images onto animated game 
players in a game program and includes an image editor to facilitate the 
mapping. Using the invention, an image of a person's face may be mapped onto 
the head of an animated game player. The person may then play the game using 
15 the animated game player having a face that appears to be that of the person. 
Similarly, images of other persons can be mapped on other animated game 
players that appear in the video game program. Accordingly, the invention 
improves the realism of video games by enabling individuals to create animated 
game players that have features, e.g., a facial image, of the individual or that 
20 were selected and imported by the individual. 

These images to be imported into the game program are not native to the 
program. The images may be taken by a digital or video camera, scanned in 
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from a photograph or otherwise captured from an external source. An excellent 
source of images is a digital camera cartridee that is inserted in the same 
machine, captures digital pictures and stores them onto a memory device 
associated with the video game machine. Typically, the captured images will 
5 be two-dimensional images of a person, place or thing, such as the face of a 
person. 

The captured digital images are transferred into a game machine running 
a game program that embodies an image editor. The transferred image is 
imported into a video editor program module of the video game program. The 

10 video editor enables the individual to display the transferred image; manipulate 
the image, e.g., crop, rotate, enlarge, reduce and adjust the contrast of the 
image; map the image onto animated game player, such as a three-dimensional 
head of such as player; change certain features of image painted on the player, 
such as changing the hair/eye color; and save the player with the mapped 2D 

15 image. The individual may also create several different animated game players 
with different transferred images. Accordingly, the invention enables 
individuals to create personalized animated players for video game play. 
During game play, the individual may select one or more personalized animated 
players for use during game play. 
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BRIEF DESC RIPTION OF THE DRA WTTVOS 

The present invention will be described in detail herein with reference to 
the following drawings, in which like reference numerals refer to like elements, 
and wherein: 

3 FIGURE 1 is- an external perspective view showing an exemplary 

structure of a game machine embodying the present invention; 

FIGURE 2 is a view showing details of the camera cartridge shown in 
FIGURE 1; 

FIGURE 3 is a block diagram showing the electrical structure of the 
10 game machine; 

FIGURE 4 is a flow chart describing the image capturing operation of 
the camera cartridge; 

FIGURE 5 is a flow chart describing the image capture, transfer and 
editing functions; 

FIGURE 6 is an exemplary video game screen image generated by an 
exemplary video game system that prompts a selection for generating an 
animated head of a game player; 

FIGURE 7 is an exemplary video game screen image that prompts a 
selection of a type of game play to which the generated animated head is to be 
20 applied; 

FIGURE 8 is an exemplary video game screen image that prompts the 
insertion of a an image capture and/or transfer device, e.g., a digital camera 
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cartridge, into the game console as a temporary replacement of a game cartridge 
with the image editor; 

FIGURE 9 is an exemplary video game screen image that prompts the 
capturing of a digital image for insertion into the video game; 

FIGURE 10 is an exemplary video game screen image that displays a 
real-time 2-D image being captured by a digital camera inserted into the game 
console, wherein the screen image includes a prompt to select, e.g., "shoot", a 
desirable real-time images; 

FIGURE 11 is an exemplary video game screen image displaying a 
selected 2-D image captured from a digital camera, and includes a prompt to 
save image or return to real-time image display; 

FIGURE 12 is a schematic diagram showing the mapping of a 2D facial 
image onto a 3D face of a 3D head; 

FIGURE 13 is a flow chart of the steps for mapping a 2D facial image 
onto a 3D head; 

FIGURE 14 is an exemplary video game screen image displaying a 
selection of 3-D heads on which has been painted the saved 2-D image, 
^ FIGURE 15 is an exemplar}/ video game screen image showing a 2D 

facial image mapped on a head and a prompt to save this personalized heads; 

FIGURE 16 is an exemplary video game screen image displaying a 
message regarding saving a personalized head in a section of video game 
system memory dedicated for user files, and includes a prompt to save the head 
in memory; 



FIGURE 17 is an exemplary video game screen image that provides a 
selection of memory locations which may include a saved personalized head; 

FIGURE 18 is an exemplar}- video game screen image that prompts a 
user to edit a completed personalized 3-D game head; 
5 FIGURE 19 is an exemplary video game screen image that shows an 

image editor for viewing a 3-D rotating personalized head in juxtaposition with 
the 2-D facial image mapped to the head, and includes prompts for editing 
features and other image commands; 

ft FIGURES 20 and 2 1 are Aemplary video game screen images of a 
10 image editor having controls tqr manipulating the shape of a 3-D head, and 

FIGURE 22 is an exemplary memory map of a portable controller pack 
with respect to the storage of personalized game player images. 

DETAILED DESCRIPTION OF EXEMPLARY EMBODIMENTS 

^QA^ ft^^ ^ * s a ^1°°^ diagram of a game sWtem according to one embodiment 
15 of the present invention. The machine is, for example, a game machine 10 

(e.g., the product "Nintendo 64") utilizing/ an advanced technology, such as 32 
or 64 bits, which is a high in processability (i.e., processing capability , for 
example, in CPU bit number, CPU program processing capability per unit time 
or image representability). The macnine uses a removable memory cartridge 12 
20 (hereinafter referred to as "cartridge 1 ') as one example of an external memory 
medium for storing game programs, and other data. 
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%l Controllers 16a, 16b, ; such as hand controllers, are electrically 
connected to the machine 10 typically via wires 1(7, although a wireless 
connection may be used. The hand controller has a joystick, buttons, keys and 



other user input devices for interactively control 
a television IS. In addition the hand controller i 



ing game play as displayed on 
icludes an electrical connector 



24 which provides a data connection between a removable data transfer pack 26 
plugged into the controller and a processor in the game machine. A transfer 
cartridge is disclosed in pending U.S. Patent Application Serial No. 09/lS9,797, 
entitled "Game System Operable With Backup Data On Different Kinds of 



Game Machines" and filed November 12, 1998, and the entire contents of 
which is incorporated by reference. The data /transfer pack includes a connector 
28 for electrically connecting to other game cartridges, a memory pack 
cartridge or to a cartridge with digital camera 14, as is disclosed in pending 
U.S. Patent Appln. Serial No. 09/430,169, Entitled "Portable Game Machine 




Having Image Capture, Manipulation and Incorporation" filed October 29, 
1999, the entire contents of which is incorporated by reference. The digital 
camera cartridge 14 may be used in connection with the transfer cartridge 26 to 
interfaces the digital camera cartridge to the machine, via the transfer pack and 
controller. An advantage of inserting the ^digital camera cartridge into a hand- 
held controller is that a user may easily align the field of view of the camera 
with a desired subject simply by holding land positioning the controller and 
camera. In addition, a digital camera cartridge may be inserted into a connector 



of the controller, while at the same time 



a s;ame cartridge 12 having an imase 



editor and game program may be inserted in a cartridge connector of the 
machine. In addition, the digital caiera may be inserted into a first hand 
controller 16a, and a second hand cJntroUer 16b may be used to input user 



control information for capturing, e 
to a memory pack (not shown in Fi<: 



sailing and a saving an image and to connect 
:ure 1) for storing data regarding the imaae 
and associated personalized 3D heab of an animated game player. 

The game machine 10 may be a console having one or more processors 
for executing game programs downloaded from a game cartridge 12, for 
generating graphical images and sounds related to game play, responding to 
10 player commands entered through hand controllers 16a, 16b, and displaying the 
generated images and sounds on a display 18, such as a television in a manner 
providing interactive game play between the displayed images and sound, and 
the user's inputs to the hand controllers, 
^yj A <"7 FIGURE 2 is an exploded showing the structure of a camera cartridge 
15 14. The camera portion is integrally formed on the upper portion (upper edge) 
when viewed from the front of the housing 50a, Lb, which contains an 
electrical circuit having electronics to support thi digital camera, memory for 
storage of 2D digital images and a connector sj for coupling the camera 
cartridge to a game machine and data pack. We housing is formed, for 
20 example, in a flat, vertically elongated rectaniilar parallelepiped shaped having 
a prescribed thickness. The connector 54 is lovided inside an opening in the 
cartridge housing 50 a, b, opening for contacting with a terminal (e.g., edge 
connector) of a substrate 52 included in the/cartridge to electrically connect a 
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circuit on the substrate to an electric/ circuit in the housing. The camera 
cartridge 14 includes a generally spLncal camera housing 20a, 20b containing 
an image pickup device, e.g., lens L and CCD 30, and is supported rotatably 
(in a lateral direction when viewei from the front by the supporting portion) on 
5 a mounting 32a, 32b on the housing of the cartridge. 

Preferably, the supporting camera housing has a rotation angle range of 
approximately 1 80 degrees so that a viewing range of the camera portion can 
turn to both of the forward direction of the front surface and the backward 
direction of the rear surface. Further, a mount angle (i.e., an angle which is a 
10 mount axis, which is parallel to an optical axis, of the camera portion tilts with 
respect to the direction at a right angle to one main surface and the other main 
surface of the housing) is selected in a range of 10 to 45 degrees (preferably, 
approximately 15 degrees), for example, which is in a upward direction with 
respect to a depth direction (a direction going through at a right angle from one 
15 main surface to the other main surface) of the housing. When the user holds the 
controller, data pack and camera cartridge with both hands and when the image 
pickup device points to the forward direction of one main surface, the face of 
the user who watches the image display device is received by the camera 22, 
and in reverse, when the image pickup device points in a backward direction 
20 the face of a person ahead of the user is received by the camera 22. 

In this m^ner, the supporting portfon rotatably supports the camera 
portion in a lateral direction of the body/jortion to allow the user to easily 
change the shooting range by simply hanging the direction of the camera 
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portion and also to shoot and display thaf forward direction of Che user and the 
user himself/herself (or the backward direction of the user) on the image 
display device when the rotation range is selected to approximately 180 
degrees. Further, the supporting a;as with which the supporting portion 
5 supports the camera portion is inc/ined to allow the user to freely adjust the 
shooting range in the forward direction or on the user side by only slightly- 
including both wrists when holctfng the housing with both hands, without 
requiring the user to take an uncomfortable position when adjusting, thereby 
and reducing the fatigue of the wrists. 
10 FIGURE 3 shows a block diagram of exemplary circuits for a game 

machine having a removable digital camera cartridge, removable data transfer 
cartridge that is used to capture 2D images and temporarily store those images 
in memory in the cartridge, e.g., RAM and removable game cartridges. 

The game machine 120 includes a CPU itl that is connected to an 
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15 input/output interface 122 (hereinafter referred to as ""I/O"). To the I/O are 

connected to connectors 123b, 123d for connection with a hand controller 124. 
Where the game machine 120 is usable for a game in which a plurality of 
players participate at a same time, a plurality of sets of connectors and 

controllers are provided. The hand controller 124 may also include a 

» / 

20 connection 150 to a portable memory pack 115, having a read/write memory for 

storage of user data, e.g., facial images, to be used during game play. The 
memory cartridge connectable to the hand controller incorporates, by mounting 
on a substrate, a non-volatile memory (e.g., ROM, EP-ROM; hereinafter 
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referred to as "ROM") for storing image files, editing files and identification 
codes. 

Another connector 123a is disconnectably connected to the game/camera 
cartridge 125. Further, the CPU is connected to a RAM 126 employed as a 
5 working RAM and an image processing unit (RCP) 127. The RAM includes 
memory areas respectively utilized for a plurality of players, and further 
including a flag area, a counter area, and register areas. The RCP is connected 
to a video RAM 128. The video RAM has a memory area for storing red (R), 
green (G) and blue (B) color data corresponding to each dot of one CRT screen 
10 140 such as a raster scan display, so that color data is written in and/or read 
from the memory area for color display under control of RCP. The color data 
read from video RAM is converted by an encoder 129 and/or composite image 
signal generating circuit into analog R, G, B signals and/or a composite image 
signal which is supplied to the CRT. 
1^£ $r^) FIGURE 4 is a block diagram of the components for capturing an image 
of a subject 300, e.g., a face of a person. Light reflected off of the subject 
towards the digital camera cartridge 304 and is focused by a lens 302 on a 
charged-coupled-semiconductor device (.CCD) 306. The CCD generates 
electrical signals representing an image /of the subject. An analog-to-digital 
20 converter 308 converts the electrical signals into a digital image of the subject. 
The digital image is formatted by anoiher converter 310 into a digital file that is 
stored in a RAM memory 312 in the/camera cartridge 304. The memory 312 
may save several digital image file^representing pictures of different pictures. 
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In addition, these image files (An be written into memory over existing files, so 
as to erase those overwritten/image files. 

The camera cartridge 304 may be inserted into and electrically coupled 
to a data transfer pack 314 to transfer the digital image files into the memory of 
5 the game cartridge 316. The transfer pack is a digital coupling device for 
transferring data and program files from one game cartridge (which may be 
normally incompatible with the game machine) into the RAM memory of the 
game machine so that those files may be used during game play. The transfer 
pack 314 may connect to a controller 318, e.g., hand controller, of the console 

10 3 16 of the game machine. The game machine console 316 may also have a 

removable game program cartridge 320 inserted into a connector. The software 
of the game program is downloaded from the cartridge into the memory of the 
game machine during game initialization. The processors in the machine 
execute the game program, which includes image capture and editing programs. 

15 These programs display the image of the subject on a display 322. 

FIGURE 5 is a flow chart of a image capture/edi ting/storage computer 
program (stored in the game cartridge 12 and downloaded into RAM 126 
during game initialization) for capturing a image of a person's face, mapping 
the two-dimensional (2D) image onto a face of a three-dimensional (3D) head 

20 of an animated game player, editing the facial image and saving a personalized 
3D head of an animated game player. The program may be included in a video 
game program to enable users to create personalized animated game players for 
game play. A root menu, step S50, of the program provides a starting point of 




the program and generates a screen menu (Fig. 6) that prompts a user to load a 
saved personalized head from memory for game play, create a new personalized 
head or edit a saved head prior to game play. 

When a user selects generation of a new head, step S52, the program 
5 may generate a screen menu (Fig. 7) asking the game play type in which the 
head will be used, e.g., player vs. player combat simulations, single player 
mission game play, or both. This selection of play type, S54, may useful where 
the game program uses, e.g., allocates different memory locations, different 
animated players for different types of game play. Once the type of game play 

10 has been selected for the new head, the program prompts, step S56, the user to 
load a two-dimensional (2D) image of a facial image to applied to the face of 
the head being created. The user is invited to insert a camera during the "Insert 
Camera Prompt" (Fig. 8). At this stage the transfer pack is inserted into one of 
the game controllers 16a, and the camera 14 is inserted into the transfer pack 

15 26. The user is prompted to select either: (i) To capture a live image using 

"real time" data that the camera is receiving, or (ii) to search the contents of the 
camera memory for images that have been stored when the camera has been 
previously used in the digital camera cartridge (Fig. 9). 

The camera may be used to take pictures of the face of the user, where 

20 one or more of the pictures are to be chosen to be applied to the animated head 
being created. During the image capture step S58, a screen is displayed 
(Fig. 10) that shows the camera image 1002 currently being viewed by the 
camera. The camera may have functions to steady the image, and to 



automatically adjusting the brightness and contrast of the image to compensate 
for background lighting. 

^ ft ^During image capture, an oval template^ is displayed on the screen 

to prompt user to center his face in image capture area. During the "Take 
5 Picture" menu (Fig. 10) the current vis^l image as being received by the 

camera is displayed on screen, and/updated by the camera at around 2 times 
per second. A blue transparenybverlay template 1004 highlights a vertical oval 
area of the image which is ^feemed to be the ideal position for the user to 
position his or her faceVFace images that fit correctly with this oval will need 
10 little fine tuning and/s such the whole capture/editing process is speeded up by 
properly position/g the user's face in the oval. The user moves before the 
camera so thayhis head is centered in the oval template shown in real-time on 
the screen. 

Showing the oval template during image capture, enables the user to take 
15 a better picture of himself and reduces the need for subsequent editing of the 
captured image. When the user is satisfied with the real-time image of himself 
(or other person or scene) being displayed within the oval template, the user 
activates a key on the hand controller to capture the image S60. The hand 
controller is the user input device through which the user selects options 
20 presented on the display screen and to otherwise interact with the program. The 
hand controller used to select and image need not be the controller supporting 
the camera. 
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After selecting an image to capture, the processor captures the image by 
storing the 2D image in memory, such as the image capture memory of the 
camera or a memory location in the game console. The captured image is 
displayed on a similar screen (Fig. 1 1) as was used for image selection. The 
5 captured image may be selected S62 or deleted so that the screen again displays 
a real-time image of the camera view so that another image may be selected. 

^jjtf A ^ A faCiai image may aIs ° be seIecte Ay choosing one of the stored 2D 
images in the camera cartridge (Fig. 9) o/by choosing an personalized 3D 
animated game player stored in a gamefoack or other memory device coupled to 

10 the game pack, controller or machine. /if a personalized 3D head is selected 
(S64), the user may be first prompted to identify the game mode, e.g., multi- 
player combat, single player missioAs or both, in step S66. The user is then 
presented on the display with the 3D personalized heads for selection in S68. 
The user may then begin game p/ay or edit (S70, S72, S74) the existing head. 

15 In addition, the user may select an existing 2D image, such as those 

stored in the digital camera in S76 from a selection of images that are stored in 
the camera, S78. These pre-existing images may have been taken by the user 
while outside or some other location remote from the game console, or may 
have been taken of the face another person, animal or thing, e.g., the face/grill 
20 of a car. 

^ytl^Once a pre-existing image is selected, tj/e image is transferred from the 
camera memory to memory in the game machine, in step S80. Regardless of 
whether the image is a pre-existing image/or an image captured from a real- 
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time camera image, the program saves the image in temporary memory in the 
game machine so that the image may be applied tD a head and edited, in step 
S82. The mapping of a 2D image onto the head may occur during start-up of 
each game (as the existing heads are readout of the memory pack) and, during 
the edicing process, as 3D heads and 2D facial images are being edited. The 
major portions of each 3D head are its face, hair and sides (neck and sides 
around the ears). The two-dimensional facial iiiage 1200 is mapped to the face 
portion 1204 of the 3D head 1206, as is shown In FIGURE 12. The hair 1206 
and sides 1208 of the head are not mapped wiai a 2D image. The 3D face area 
1202 is an 3D generally oval surface. This face area surface is defined 
mathematically by computer software as triangles, e.g., 20 triangles, that when 
assembled together for a 3D surface of the face of the head. To generate a 3D 
face, to be inserted in a head that may be oriented from the front, side, looking- 
up, looking down or other point of view, the computer determines the location 
and position of each of the triangles that/define the face. The location in three- 
dimensions of the surface of the face is/determined determining the location of 
each of the mathematically triangles tlj(at form the 3D face image. 

s£ir>f ft l ^^ e 2D ^ acial ima £ e is applied to tfob 3D face by mapping the 2D image 

to the triangles that form the 3D face. For mapping, the 2D image is segmented 
20 into triangular sections that correspond t/o the 3D triangles of the face. The 
segmentation is done by identifying thnee points on the 2D image that 
correspond to the comers of each triangles of the 3D face. The section of the 
2D image within the triangle defined by each of the three identified points is 
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mapped onto the corresponding triangle ofl the 3D face. For example, one of 
the triangles of the 3D face portion may correspond to the nose of the face. To 
map the nose of the 2D image to the 3D iose triangle, three points on the 2D 
image are identified that outline the nose! shown in that 2D image. The 2D 
5 image within the triangle defined by the /three selected points are mapped to the 
triangle corresponding to the nose of the 3D face. Similarly, the other 
triangular sections 2D image are mapped to the other triangles that makeup the 
3D face. The mapping of the 2D imagp onto a 3D face is accomplished each 
time the video game starts up and duriing editing. 
10 FIGURE 13 shows a flow chart for mapping a 2D image onto a 3D face. 

In step S1300, the parameters of the oval 2D facial image (see 1200 of Fig. 12) 
are obtained from the stored image. The face 1202 of the 3D head 1204 is 
aligned in 3D such that the 3D face is aligned with a 2D plane that is 
orthogonal to the z-axis. In particular, the vertices that define the facial surface 
15 are oriented with respect to the 2D plane such that the vertices are evenly 

spaced on either side of the plane, S1302. In step 1304, the center point 1210 
of the 3D face is determined as follows: 

With reference to the 3D model, the center of the face is determined by 
equation (1): 
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n r=0 71 1=0 



wherein the nomenclature for the above equation (1) is: 
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n - number of vertices in the face part of the head model. 

VX| - X co-ordinate of the rth vertex 

VYj - Y co-ordinate of the i'th vertex 

(s,t) - position of center of the oval in x and y respectively 

w - width of the oval (in X direction) 

h - height of the oval (in Y direction 

In step 1306, the far edges of the 3D facial images are determined by the 
following equations (2) and (3): 

X m ax = Maximum distance in x direction of any vertex from the center of 

the face X cen (equation 2) 
Y ma .x = Maximum distance in y direction of any vertex from the center of 

the face Y cen (equation 3) 

The transformation scalars, SI 308, that convert the position of a 3D face 
vertex (xyz) into a corresponding 2D co-ordinate on the face image may also be 
referred to as a texture co-ordinate. This position dictates the corresponding 
position in 2D space that the 3D-vertex position should represent. 



X^i^ = w/ (2 * X max ) Equation 4 

Y SC aiar = h/ (2 * Y max ) Equation 5 

-T^i A ' ca l cu l ate r he transformed (texj?6re) co-ordinates equations 4 and 5 are 

25 applied to each vertex, S1310, £1312, S1314 and S1316. 

The i'th transformed vertex has co-ordinates in X and Y and these are 
defined as the U and V texture co-ordinates whereby: 
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Ui = S + (VXi - X.,, ) * X scaJar Eq . 6 

V, = t + (VY, - Y a ) * Y scllar Eq . 7 

5 Ui and Vj are included m the 3D information of the face which is used to 

draw the head with the relevant areas of the 2D image mapped onto the 3D 
triangles that make up the face model. 

^ /V ti/ 1 ^ stored 30 " h " ds " which the user se^cts are stored in the game 

program (or memory controller pack) as ^s" S82. When completed, the one 

10 slot contains all the information to generate a custom character head this 
includes face texture information/air and face colors, head type etc. The 
game allows multiple slots fo^rnultiple heads that can be used m the single 
player version of the g arr/where the custom heads can be used for either your 
player character' sjie^d or that of a random guard or other computer-controlled 

15 game char actei 

The customer heads, i.e., those generated using a 2D facial image and 
the image editor, can be used throughout game play for those animated players 
controlled by individuals and for the computer generated animated players. In 
multi-player mode, there may be more head slots, e.g., twelve, to be stored 
20 because there is the potential of multiple memory packs in each of the hand 
controllers being used by the multiple players. 

When loading files that are previously saved S64, S70, on the memory 
packs a menu appears whereby icons representing the stored files are displayed 
on screen, FIGURE 14. The icon image of each head 1400 is a reduced size 
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version of the captured image. The user selects an image using the controller 
and hits the trigger to load the head. The data that is loaded from the memory 
pack includes a compressed version, e.g.. JPEG compression, of the original 
image taken from the camera and all other parameters set up by the user; these 
include head type, color, etc. 

Once a facial image has been selected/the user is prompted to apply the 
image to an animated three-dimensional head, in step 568 and shown in 
FIGURE 15. The machine may have scored several, e.g., five, animated 3D 
heads which are used to create animated game players. These heads may be 
male or female, and have different shapes, hair color and skin tone. The heads 
are presented on the display as individual 3D head images. The heads may 
having blank faces, a computer generated face or have the face of the 2D facial 
image that has been captured and selected for use in creating a personalized 
head. The heads may move, e.g., turjn from side to side and tilt up and down, so 
that the user may see the head from/different angles. The user selects a head on 
which to apply the facial image that as been captured. 

Once a head is selected, the program maps the 2D facial image (as 
framed by the oval template) on the face portion of the selected head. 
Alternatively, the computer may have automatically mapped the image onto 
each of the heads displayed for selection. The head selected by the user is 
saved, in step S82 at a location in memory that may be selected by the user as is 
shown in FIGURES 16 and 17. The user is given the option to save S84 the 
current head information to a controller pack so that the head can be easily 
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retrieved at any time. It may be the user only wishes to use the head in this 
instance of the game or that he or she does not own a controller pack. If either 
of these is the case then the head need not be saved. 

The saved head is ready for game play and may be applied to the body of 
5 an animated game player so that play may begin in step S84. At this stage all 
the information is available to display a custom head. Various options will still 
be in their default modes but these may be changed at a later time using the 
editor menu, FIGURE 18, and step S74. The editor menu may display both the 
2D image of the face 1800 that may include a translucent oval template to 
10 indicate the portion of the image that is being mapped to the 3D head 1802, that 
is also shown in the menu. In addition, the menu may display a list of editing 
commands 1804 for use in editing either or both the 2D image 1800 or 3D head 
1802. 

A* ^ ^° an an i matec i 3D head wip a mapped 2D facial image, the user 
15 selects one of several available editing functions, including selecting a new 2D 
facial image S86, select a new 3D head S88, reposition the 2D facial image 
S90, adjust the 3D shape of the head S92, change the hair color and skin tone of 
the head S94, delete the saved head S96, cancel any changes S98 that have been 
made to a saved head during thfe editing process, and save any changes made to 
20 the head and facial image S U)0. 

In particular, the editor (zontains various sections to alter the 
characteristics of the current/head, these include: 
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Take Another Picture (S86) 
This option leads to the 'Take 
change the image that is mapp 



Change Head (S88) 

The "Change Head" function 



the special face mapping enabled character heads. 
Position Picture (S90) 
This menu allows the user to £ 
are mapped onto the head's f 



Picture" menu allowing the user to 



:d on to the head's face. 



allows the user to alter his or her choice of 



hange which parts of the captured image 
.ce. Five control handles allow the size, 
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shape and position of the aforementioned oval area to be altered, as is 



shown in FIGURE 19. The 
this oval on to the facial aret 



helping to disguise the joint 



software maps the 2D image that falls inside 
of the head. An additional "Brightness" 



feature allows the user to change the brightness of the facial image 



between the face and the rest of the head. 
15 toTThe editing by the user of the capthred 2D image S90 includes, adjusting 

the position of the oval template over the 
the oval shape about the major and minor 
and adjusting the brightness of the 2D inj[; 
template 1902 defines the portion of the 
20 3D face. The template is translucent so pat the entire 2D image (both inside 
and out of the template) may be viewed' which facilitates the positioning of the 
template on the 2D image. The templa 



2D image, expanding and contracting 
axes (length and width) of the oval, 
age 1900 in Fig. 19. The oval 
2D image that will be mapped to the 



image by use of cursor commands and irag points that are on the rim of the 



:e can be moved with respect to the 2D 



oval and chat can be manipulated byithe user with the hand controller and a 

cursor that appears on the image in figure 19. 
^55 Moving the template with r^pect to the image allows the user to better 

aligns his face in the oval template? and, thereby, adjusts the alignment of the 
5 facial image on the 3D head 1904. Expanding and contracting the length and 

width of the oval also allows a i^iser to better position his facial image within the 

oval and, thus, onto the 3D facj 

The editing of the 2D image is done with a screen display of the 2D 

image in the oval and a mapped 3D image, which shows in real time the effect 
10 of editing the 2D image. In addition, the brightness/contrast 1906 of the 2D 

image may be adjusted during 2D image editing. 

The oval 2D image becomes distorted when mapped to the 3D face, 

especially as various sections of the 2D image are deformed to fit the triangles 

that make up the 3D face. Allowing editing of the 2D image allows the user to 
15 imprr ve his 3D facial image by adjusting the 2D image before it is mapped. 

The still image editor displays the 2D image and the 3D image (with limited 

rotation of the head) during the editing process so that the user can view the 

ultimate 3D image of his face while editing the 2D image. 

Pr' 1 ^/ The 2D ima ge editor may also automat/cally balances the contrast of the 
20 image to reduce the unintended effects of shAding on the 2D image. These 

shading effects can cause poor 3D visual eflrects when the 2D image is mapped 
on the 3D face. To balance the contrast, tMe image editor compares the 
brightness of the left side of the image to/the right side of the image. If the 
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brightness between the two sides varie^ excessively, such as by more than 10%, 
then the editor reduces the brightness ©n the bright side of the image and may 
increase the brightness on the dark sice of the image. In addition, the 
brightness/darkness adjustments are more pronounced at the edges of the oval 
image than at the center because the brightness adjustments are applied in a 
linear manner in which the center of /the image is not adjusted so as to avoid 
creating a perceptible contrast change at the center of the image. 
^ When using a digital camera tal/es an ima.ee in ''real" world conditions, 
it is often found that the brightness oi the image is not consistent across the 
image when measured from left to right. This can create problems with face 
image mapping as the brightness on the join line at each side of the head is at 
the same brightness. The join linef is the 3D surface line where the face portion 
of the head meets the sides of the? face. If there is a difference in brightness at 
the join Line, the result is a distinct and unwanted shade discontinuity on one or 
15 both sides of the mapped head. 

^7 In order to combat this effefct "Automatic Shade Compensation" 
techniques are used during the mapping phase of the operation. This attempted 



to modify the brightness of the 2 
mapping without unduly affectir 



captured image to give a more even 
g the "look" of the overall image. 
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^ This is achieved by applying a r^6n-linear filter to the 2D image, which 
modifies the intensity of each pixel asfa function of pixel position and based 
upon the following equations. This filter is applied to the whole image, 
however only pixels within the ovaljare used to calculate the filter parameters 
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N x = Number of pixels in X direction in the 2D image 
N y = Number of pixels in Y direction in the 2D image 
^£ fcl&J Nsummed = Num ber of pixels used in the summation (i.e. Fall with 
5 25% and 75% bounds) 

I xy = Original intensity of pixel at position (x,y) 

I xy = Transformed intensity of pixel at position (x,y) 

Au-ft = Average intensity left of center of the image 

A R ig ht = Average intensity right of center of the image 

10 s = Center of the selection oval in x direction 

t = Center of the selection oval in y direction 

w = Width of selection oval (x direction) 

h = Height of selection oval (y direction) 

15 An intensity value is calculated for each side to generate a measure of 

the balance error in the current image. In this regard all intensities may be 
normalized (i.e. Range from 0.00 (Black) to 1.00 (White)). The average 
intensity is calculated for the left and right sides of the image using all pixels 
that fall between 25% and 75% of maximum intensity. This helps to reduce 

20 errors caused by small artifacts that have extreme intensities. 

1 ^ 1 .r=j+(>W2)y=<+(n/2) 



U/, = N £ 2X(°-25 </^<0.75) A,„ A ,=_i_ £ £7^(0.25 </„< 0.75) 

summed x—s~iw / 2) v={-(h / 2) /V ' ^ *> J 

summed x=x y~t 
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Shade compensacion is used if [A Lefl - ARi ght | > 0.05 
If shade compensation is to be added then the following filter is applied 
to the image for pixels left of the center of the image: 

C = ^ * (1 + (Mr - (2 * x)l Nx ) * 0.5 * ( A Rlghg - A Uft ))) 

5 

For pixels right of the center of the image: 

C = * U + ((((2 * .t) - Nx)) I Nx) * 0.5 * (A Ufi - A WjA , ))) 

10 

Note that at x=N/2 no intensity changes occur on either side thus 
preventing a discontinuity appealing at the center of the image. 

15 Color Face & Hair (S94) 

The face and head colors can be each changed to one of four pre- 
determined types. The user is free to pick any combination of these to 
suit his or her preference. 
rTT) Shape Head (S92) i 

20 The "Shape Head'? feature allows the user to alter the three dimensional 

shape of the curreht head, as is shown in FIGURES 20 and 21.. Control 
points allow the nead to be squashed and stretched in dimensions 
coplanar to the face and an additional slider allows the depth of the head 
or scale in the front to back direction to be altered. 

25 The 3D entire head is manipulated, including the face, side and hair. 

The color of the skin tone and hair can be selected using the editor. In addition, 
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the shape of the head may be changed via a screen (FIGURES 20, 21) that 
displays the 3D head 2000 (in real-time and showing different viewpoints of the 
head) juxtaposed next to a box 2002 that is a tool for manipulating head shape. 
The box has eight (8) points, located at its four corners 2004 and 
5 midway 2006 between pairs of corner, that can be dragged using a cursor and 
hand controller to change the shape of the box and, thus, change the shape of 
the head. A comparison of Figures 20 and 21 show that by deforming the box 
(see Fig. 20) the head 2000 is similarly deformed. By adjusting the shape of 
the head using the box tool, a user can reshape the head to better conform to the 

10 real-life head shape of the individual whose facial image is being mapped to the 
head. Alternatively, the box tool may be used to create a cartoon shaped head 
or to generate other special effects with respect to the head. 

Using the 3D head manipulator, the head can become wide and squared; 
narrow and oval like, or many other shapes. In addition, a slider bar in this 

15 screen stretches or compresses the front-to-back dimension of the head. In 

addition, by adjusting the slider bar 1906, the head may be stretched from nose 
to back, to as to generate a head that is long (nose to front) and narrow (side-to- 
side) or, alternatively, a head that is short (nose to front) and wide (side-to- 
side). 

20 Other editing functions include: 

Removed Head (S96) 

Removing a head clears the slot that is currently being edited; all 
changes are lost. 



Cancel Changes (S98) 

Canceling changed returns the head to that state that it was in prior to the 
editor menu being opened. 
^$■#^7 K eeP Changes (SIOC/) 
5 Keeping changes ur/dated the current selected slot with the alterations 

made during the editing session. Following this the user is again invited 
to save his or her lork to a controller pack. 
frW The personalized heads that were created usina the imaae 
capmre/edit/save functions may be used duing game play. Typically, a user 
10 will select a personalized head that includ/s the user's facial image and apply 
that head to the body of a desired anima Jd game character. In this way, users 
can personalize game play by having the/ animated player that they control be a 
player that has the appearance of the us/er. During multiple-user combat game 
play, it would enhance the realism of lame play if the animated game players 
15 that appear in multi-player game mofle. 

^7 Moreover, the personalize heads ma(y be applied (in a random manner or 
some prescribed manner, e.g., all guards Kave a certain head) to computer 
controlled game characters, such as gua/ds that appear in a combat game, where 
the guards are not controlled by any o/the game playing users. During game 
20 play, any heads that occupied the "Single Player Slots" are randomly selected 
and used as the heads for that parti/ular levels guard characters.. Thus, the user 
will see personalized computer generated players as he plays a game. In 
addition, if the slots set aside fo/ multi-player games contain any valid 
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personalized heads then these heads may appear (Fig. 14) as a head option 
when the player chooses to modify his character's properties. Thus, the player 
may use or edit any of the saved "heads" in/the controller memory pack. 
^ The 2D image and settings made w|6i the editor are stored to save a 
personalized animated game player. The£D image and settings may be stored 
in the user's controller memory pack so /hat they may be carried by the user 
from one game console to another. The/ memory pack plugs into the hand 
controller and may be carried by the usL from game machine to game machine 
and, thus, allows the user to load his personalized 3D animated game player 
(including a face that is derived from I 2D image of the user) into any game 
machine that the user may be playing. Thus, some advantages of usin* 



controller game packs for storage of 
carry personalized player settings to 



personalized player head is that user can 
Other game consoles, especially those 



consoles' owned by friends. Other advantages of a portable personalized player 



head stored in the controller memorv 
other where each friend has their owfa 
images may be formed for the comp 
generated guards and other "bad zm\ 



pack include that friends may play each 
personalized player, and personalized 
iter controlled players, e.g., automatically 
s M to randomly have personalized heads 
appear on animated figures during the game. 

As shown in FIGURE 22, the data file stored in the memory pack 
corresponding to one personalize head (i.e., the 2D image and head settings) 
1900 includes: compressed 2D image 1902; settings of oval shape and oval 
location, e.g., center point of oval 1904; head, hair color and skin tone setting 
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collectively 1910; setting points of the eight (8) points of the 3D manipulator 
section 1906; brightness level 1908, and a unique identifier 1912 for the head. 

The saved heads, including 2D facial image, may be compressed to 
conserve storage space in the memory pack. A compression technique similar 
to the well-known JPEG technique may be used. Such a technique would be 
embodied in a software program included in the image editor program. To 
compress the slot (head file), a smoothing filter to the image to eliminate sharp 
contrast gradients in the image. The image is then divided into blocks, such as 
8x8 pixel blocks. A discrete Cosine transformation is applied to each block of 
the image. A quality controller program may be applied to the blocks defines a 
compromise between image quality and image file size that is applied to 
compress the image data without unacceptably deteriorating image quality. The 
data is re-ordered in a zig-zag pattern to improve compression potential. Then 
that data is processed by a run length encoder compression scheme and a bit 
Dack compression scheme to create a compressed data file of the image. 
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Detailed Implementation Example 

Roots though the system: Description 

Root Menu (VTC = 00:01:08 ) 1 

The Root menu if the scan point for the user to access the Face mapping technology in Perfect Dark. 
Load a saved head (VTC = 00:01:11 ) 

This section allows the user to load data previously created into the svstem. This information is stored 

'hartrr/T ,ns , a r as '■ iiots "- one * an the ^-0™™ » «sr a ciSi 

character head, th.s mcludes race texture information, ha.r and face colours, head type etc 

The game allows six slots to be head at one time that can be used in the sinsle olaver version of the 
game where the custom heads can be used for either your player character's" head or that of a random 

caused ttv'^V' ^ ^ ^ ^ ^ Sl ° tS 10 be SCOred at ^ one time. These can 
beused by any or the players pamc.pat.ng ,„ a mult-player game and are used as that players character 

^^■::^:^r the slots avaiiabie for ^ ° r - ^ « » > 

Load Perfecthead (VTC = 00:01:15 ) 

When loading files that are previously saved on the memory packs a menu appears whereby icons 

ca^u S imale^rV °" ^ The 1C °" ima * * a red "« d «" versi^ of he 

Zr , S h 3 eCtS ^ ' mage US ' ng the contro "er and hits the tri a2 er to load the head The 

data that ,s loaded includes a compressed version of the ordinal imase taken from the camera and all 
other parameters set up by the user; these include head type! colour. e\c. 

Make a New Head (VTC = 00:01:24) 

To make a new head the user firstly needs to grab an image from the camera, which will be used as a 
game character's face. As in the previous sect.on the resultant head can be u ed i n d he or both the 
single and multi player areas of the game. 

Insert Camera Prompt (VTC =00:01:24 ) 

?r h a e nSe e rPnr ited * "'V* the " InSerT Camera At this stase the 

Transfer Pack >s mserted mto the Game Controller and the Camera then inserted into the Transfer Pack. 

At this stage the user has two options : 



1. 
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To capture a live image using "real time" data that the camera is receiving 

To scan the contents of the camera for images that have been stored when the camera has been 

previously used m conjunction with a Nintendo Gameboy 



Take Picture (VTC = 00:01:31 ) 

During the 'Take Picture" menu the current visual data as being received by the camera is displayed on 

STma-ewhich d * T"! * ^ * ° ,Ue 0Veday * VerticaI oval «a of 

he .mage which .s deemed to be the .deal position for the user to position his or her face. Face imaoes 
that fit correctly w.th th.s oval will need little fine tuning and as such the whole processes speedTd up 

^ USer iS ha PPy with the ^rrent image the shoot button freezes and stores the ima*e after 
wh.ch the user can continue with the process or retry the capture. 



^^l^M^t °' relCVam inf ° rmati0n 35 ShOW » °" ** -ompanying VHS 
" The multi-player mode of the game can be referred to as the "Combat Simulator" 
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Choose Head (VTC = 00:02:04 ) 

Special character heads where developed chat allows the system to add a different facial image. At this 
stage the user chooses a head whose shape and hairstyle best suits his or her requirements. 

Save Perfecthead (VTC = 00:02:11) 

At this stage all the information is available to display a custom head. Various options will still be in 
their default modes but these may be changed at a later time using the editor menu. 

The user is given the option to save the current head information to a controller pack so that the head 
can be easily retrieved at any time. It may be the user only wishes to use the head in this instance of the 
game or that he or she does not own a controller pack. If either of these is the case then the head need 
not be saved. 

Select Location (When saving) (VTC = 00:02:13) 

If the user is saving a head, the system displays information on any controller packs that it finds that 
have the adequate amount of free space. The user can also delete previous files to free up room should 
he or she require to do so. 

Head Complete (VTC = 00:02:18 ) 

The ''Head Complete" menu inform the player that enough information has been gathered to used the 
head inside the game but invited him or her to edit various features of the head inside the editor feature. 
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Perfect Head Edixor (VTC = 00:02:22) 

The editor contains vanous sections to alter the characteristics of the current head, these include : 
Take Another Picture (VTC = 00:02:25 ) 

This option leads to the 'Take Picture' menu allowing the user to change the imaee that is 
mapped on to the head's face. 

Change Head {VTC - 00:02:40) 

The "Change Head" function allows the user to alter his or her choice of the special fac- 
mapping enabled character heads. 

Position Picture (VTC = 00:02:45 j 

This menu allows the user to change which parts of the captured imaae are mapped onto the 
head s face. Five control handles allow the size, shape and position of the aforementioned 
ova area to be altered. The software then mapped image data that fails inside this oval on to 
the facial area or the head. An additional "Brightness" feature allows the user to change the 
brightness or the facial image helping to disgu.se the joint between the face and the rest of the 

Colour Face & Hair (VTC = 00:03:33) 

The face and head colours can be each changed to one of four pre-determined tvpes. The user 
is tree to pick any combination of these to suit his or her preference. 

Shape Head (VTC = 00:03:44) 

The "Shape Head" feature allows the user to alter the three dimensional shape of the current 
head. Control points allow the head to be squashed and stretched in dimensions coplanar to 
the face and an additional slider allows the depth of the head or scale in the front to back 

direction to be altered. 

Removed Head) 

Removing a head clears the slot that is currently being edited; all changes are lost 
Cancel Changes 

Canceling changed returns the head to that state that it was in prior to the editor menu bein* 
opened. ~ 

Keep Changes (VTC = 00:04:14 ) 

Keeping changes updated the current selected slot with the alterations made durins the editin- 
sess.on. Following this the user is again invited to save his or her work to a controller pack. * 

Edit a Loaded Head (VTC = 00:04:20 ) 

Selecting "Edit a Loaded Head" allows the user to choose from any of the currently active slots in the 
game s memory. On making this choice the relevant head is Ioaded.into the editor and chanses made as 
described in the aforementioned section. 
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Using the heads in game: - Description 



Single Player Game 

In version 4.3 of Perfect Dark any heads that occupied the "Single Player Slots" are randomlv selected 
and used as the heads for that particular levels guard characters." 

Multi-player Player Game (VTC -00:04:37) 

If the slots set aside for multi-player games contain any valid heads then these heads will appear as a 
head option when the player chooses to modify his character's properties. An example of this head 
selection can be found at VTC = 00:04:50. 
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Additional Information: - Saving data and file formats 

When a player saves Perfecthead data to a Controller Pack the image data needs to be compressed both 
to allow load times and file sizes to be reasonable. The compression is performed using a custom piece 
of software that :- 

• Applies a smoothing filter to the image to remove any sharp contrast gradients 

• Breaks the image up into S*S pixel blocks 

• Applies a "Discrete Cosine Transformation*' to each of the blocks 

• Processes this data with a quality controller that make a comprise between image quality and file 
size. 

• Re-orders the resultant data in a "zigzag M patter that maximizes it's compression potential 

• Applies a run length encode compression scheme 

• Applies a bit pack compression scheme. 

N.B. This system is very similar to the way in which JPEG compression is performed and results in an 
average compression ration of 5:1. 
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No me nc I a cure: 

n - number of vertices in the face part of the head model. 

VXi - X co-ordinate of the i'th vertex 

VYi - Y co-ordinate of the i'th vertex 

(s.t) - position of centre of the oval in x and y respectively 

w - width of the oval (in X direction) 

h - height of the ovai (in Y direction 

With reference to the 3D model, the centre of the face is determined by : 



n , 



t=0 



= Maximum distance in x direction of any vertex from the centre of the face X^ en 
Ymax = Maximum distance in y direction of any vertex from the centre of the face Y cen 

We define transformation scalars that convert the position of a 3D face vertex (xyz) into a 
corresponding 2D co-ordinate on the face image that is sometimes referred to as a texture co-ordinate. 
This position dictates the corresponding position in 2D space that the 3D-vertex position should 
represent. 

X*** = w/ (2 * X™, ) YscaIar = h/ (2 * ) 

To calculate the transformed (texture) co-ordinates the follow equation is applied to each vertex. 

The i'th transformed vertex has co-ordinates in X and Y and these are defined as the U and V texture 
co-ordinates whereby: 

U, = s + (VXj - Xccn ) * Vi = t + (VYi - Y cca ) * Y scllar 

Uj and Vj are included in the 3D information of the face which is used by the renderer to draw the head 
with the relevant areas of the 2D image mapped onto the 3D triangles that make up the face model. 



Automatic shade compensation 

When using [he camera in "Year" world conditions it is often found that the brightness of the image is 
not consistent across the image when measured from left to right. This can create problems with face 
image mapping as the brightness of the join line at each side of the head is at the same brightness. The 
result is a distinct and unwanted shade discontinuity on one or both sides of the mapped head. 

In order to combat this effect "Automatic Shade Compensation" techniques are used during the 
mapping phase of the operation. This attempted to modify the brightness of the 2D-captured image to 
give a more even mapping without unduly affecting the ;i look" ofthe overall image. 

This is achieved by applying a non-linear filter to the 2D image, which modifies the intensity of each 
pixel as a function of pixel position and based upon the following equations. This filter is applied to the 
whole image, however only pixels within the oval are used to calculate the filter parameters 

N x = Number of pixels in X direction in the 2D image 

N y = Number of pixels in Y direction in the 2D image 

^Summed = Number of pixels used in the summation (i.e. Fall with 25% and 75% bounds) 

Lp = Original intensity of pixel at position (x,v) 

I xy = Transformed intensity of pixel at position (x.y) 

A uft = Average intensity left of centre of the image 

A-Right = Average intensity right of centre of the image 

s = Centre of the selection oval in x direction 

t = Centre of the selection oval in y direction 

w = Width of selection oval (x direction) 

h = Height of selection oval (y direction) 

An intensity value is calculated for each side to generate a measure of the balance error in the current 
image. 3 

The average intensity is calculated for the left and right sides of the image using all pixels that fall 
between 25% and 75% of maximum intensity. 4 

A u fi =j; E EM 0 - 25 <'<>•< 0- 7 5) A.**" E EV°- 25< ^< 0 - 75 ) 

summed x=s-(\v 1 2) y=t -( h i 2) summe({ m . y=( 

Shade compensation is used if |A Uft - A Risht | > 0.05 

If shade compensation is to be added then the following filter is applied to the image 
For pixels left of the centre of the image: 

! l- =^*a + ('Vr-(2*.r)/iV.r)*0.5*(A iBlto - A Uft ))) 

For pixels right of the centre of the image: 

/I. = l v *a + «((2*x)-Nx))/Nx)*0.5*(A L<fi - A,.,,,))) 

Note that at x=N x /2 no intensity changes occur on either side thus preventing a discontinuity appearing 
at the centre of the image. 



3 All intensities are normalized (i.e. Range from 0.00 (Black) to 1.00 (White) 
This helps to reduce errors caused by small artifacts that have extreme intensities. 
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^ <ybj While this invention has beer/ described in conjunction with specific 
embodiments thereof, it is eviden/ that many alternatives, modifications and 
variations will be apparent to thtose skilled in the art. Accordingly, the 
preferred embodiments of ihd invention, as set forth therein, are intended to be. 
illustrative, not limiting. Various changes may be made without departing from 
the true spirit and full scope of the invention, as defined in the following 
claims. 
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